Unless otherwise indicated herein, the materials described in this section are not prior art to the claims in this application and are not admitted to be prior art by inclusion in this section.
In computing, the term “memory wall” describes the growing disparity between central processing unit (CPU) speeds and speeds of memory located outside the CPU. A prominent factor contributing to the speed disparity is the relatively slow performance of the memory located outside the CPU. During the last two decades, this disparity has increased, thereby exacerbating computational performance issues relating to the memory wall. Over that time period, processors have advanced from handling instructions in-order to out-of-order and utilize multiple cores. Such processing advances have put more pressure on the memory system. Meanwhile, data intensive applications have become increasingly common in diverse fields such as bioinformatics, computer aided design, and complex social media interactions. As a result, memory stall time (the time a processor is waiting for data from a memory) accounts for a significant portion of total application execution time. As such, data stall time creates a performance bottleneck in many computing systems. At the same time, memory systems have advanced with many concurrency-oriented features such as multiple ports, multiple banks, pipelines, and non-blocking caches, which may present an opportunity for innovation.
Hierarchical memory is a design of modern computers to ease the memory wall problem. An advantage provided by memory hierarchy is data locality, which may be described more specifically as temporal and spatial locality. That is, previously accessed data can be readily used again in future data accesses (temporal locality), and the data near the previously accessed data are likely to be used next (spatial locality).
In addition to memory hierarchy, a modern memory system is supported by various data access concurrency. The overall performance of a memory system is based on a combination of memory hierarchy and concurrency.